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STATUS OF THE CLAIMS 

1. (CURRENTLY AMENDED) A method of indexing multimedia 
documents, the method being characterized in that it comprises the 
following steps: 

a) for each multimedia document, identifying and extracting 
terms t ± constituted by vectors characterizing properties of each 
of the multimedia documents; for indexing, which characterizing 
properties include at least one of such — as- shape, texture, color, 
andei? structure of an image, an__the — energy level , an t -he 
oscillation rate or frequency information of an audio signal, 
and- e^e a group of characters of a text; 

b) storing the terms t ± characterizing the properties of each 
of the multimedia documents; in a term base comprising a set of P 
terms , where P is an integer ; 

c) determining a maximum number J_N]_ of desired concepts 
combining the moot pertinent terms t ± , where N is an integer less 
than P, with each concept c ± being designed to combine all terms t ± 
based on a relationship of that — arc neighboring from the point — of 
view of their characterizing properties charact eristics ; 

d) calculating afche matrix T of distances between the terms 
t ± of the term base of P terms ; 

e) decomposing the set of P terms - ef- — terms — fc- — of — the — term 
base into N portions P^ (1 < j < N) such that the set of P terms = 
P 1 u P 2 ... u Pj ... u P N , each portion P^ comprising a set of terms 
t ±j and being represented by a concept Cy the terms t ± being 
distributed in such a manner that terms that are farther away are 
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to be found in distinct portions P lf P m while terms that are closer 
together are to be found in the same portion P 1 ; 

f) structuring a concept dictionary so as to constitute a 
binary tree in which each leaf of the binary tree fe he — leaves 
contains afehe concept [ [■&] ] c i of the dictionary and each nodefe he 
nodes of the binary tree contains the — information necessary for 
scanning the tree during a stage of identifying a multimedia 
document by comparing it with previously-indexed documents; and 

g) constructing a fingerprint base made up of athe set C of 
concepts c ± representing the terms t ± of the multimedia documents 
to be indexed, each multimedia document being associated with a 
fingerprint that is specific thereto. 

2. (CURRENTLY AMENDED) An indexing mcthod A method of indexing 
multimedia documents according to claim 1, characterized in that 
each concept c ± of the fingerprint base is associated with a data 
set comprising afehe number of terms_^_ inNe-r- Tj_ in the multimedia 
documents in which said fehe concept c ± is present. 

3. (CURRENTLY AMENDED) An indexing mcthod A method of indexing 
multimedia documents according to claim 1, characterized in that 
for each multimedia document having a corresponding 4 n — which — a 
concept c ± is present , a fingerprint of the corresponding concept 
c ± is registered in the multimedia document, said fingerprint 
containing at least one of at rhe frequency of occurrence of with 
which — the corresponding concept c ± — occurs , the — identities of 
concepts neighboring the corresponding concept c ± in the 
multimedia document, and a score which is a mean value of 
similarity measurements between the corresponding concept c ± and 
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the terms t ± of the multimedia document that are most proximate 
the closest to the corresponding concept c ± . 

4. (CURRENTLY AMENDED) An — indexing mctho d A method of indexing 
multimedia documents according to claim 1, further 
comprising: characterized in that it comprises a step of 

optimizing the step of decomposing partitioning of the set of 
P ef— terms of the term base by decomposing to decompose said set 
of P terms into M classes C i ( in which 1 < i < M, where M is an 
integer and M < P), so as to reduce a the distribution error of 
the set of P -of— terms in the term base into N portions (P lf P 2 , 
P N ) where each portion P ± is represented by the terms t ± that areie 
taken as the concept c i , 

the distribution error that is committed j£] being such thatj_ 

E = ' 

wherein s ti = ^d 2 (t 1 ,t j ) is the distribution error committed by 
replacing the terms t^ of a portion P ± with the terms t H . 

5. (CURRENTLY AMENDED) An — indexing mcthod A method of indexing 
multimedia documents according to claim 4, characterized in that 
it comprises the following steps: 

i) decomposing the set of P ef — terms into two portions J_P X 
and P 2 j_; 

ii) determining a first termfe he — few© — terms t ± and a second 
term tj of the set of P terms that are fehe — fau-rthest apart, 
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corresponding to afche greatest distance D ±j of the distance matrix 
T of distances ; 

iii) for each term t k of the set of P terms , examining to see 
whether athe distance D ki between the term t k and ef- the first term 
t ± is less that athe distance D kj between the term t k and the 
second term t jA and if so, allocating the term t k to the portion 
P x , and otherwise allocating the term t k to the portion P 2 ; and 

iv) iterating step Ji) until at-he desired number N of 
portions P ± has been obtained, and on each iteration applying the 
steps J_ii) and J_iii) on the first and second terms of the portions 
P 1 and P 2 . 

6. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 4, further 
comprising; characterized in that it includco 

optimization starting from N disjoint portions {P lf P 2 , P N } 
of the set of P terms and N terms {t lf t 2 , t N } representing them 
in order to reduce afche decomposition error of the set of P terms 
into N portions, further comprisinga ftd — ift — that — it — comprises — the 
following steps : 

i) calculating a corresponding the centers- of gravity C ± for 
each of the portion [[>]] P ± ; 

ii) calculating a corresponding decomposition error using: 

errors eC ± = JV^^h 

tjePi 

replacing the terms t^ by center of gravity terms ; and 

calculating a corresponding distribution error using: 
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eti = J]a 2 (t ± ,t.) 

tjEPi 

whcn by replacing the terms tj of the portion Prospectively by G± 
and by the terms t H ; 

iii) comparing £t ± and £Oe i and replacing the terms t^ by the 
center of gravity terms if eeC ± < et ± ; and 

iv) calculating a new distance matrix T between the terms t ± 
of the term base and the process — &f— decomposing the set of P of- 
terms of the term base into N portions, unless a stop condition is 
satisfied with 

ec. -ea.., , , , , 

— — < threshold, 

ec t 

where Ec t represents the decomposition error committed at instant 
t . 

7. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 1, wherein characterized 
4rPt — that — #©^f — fcke — purpose — ef- — structuring the concept dictionary 
includes : — 

producing a navigation chart is produced iteratively on each 
iteration, beginning by splitting the set C_of concepts into two 
subsets, and then selecting one subset on each iteration until 
athe desired number of groups is obtained or until a stop 
criterion is satisfied. 

8. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 7, wherein characterized 
in that the stop criterion is satisfied when is constituted by the 
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fact — that each of the two subsets obtained isa^e — a±i homogeneous 
and has a with small standard deviation. 



9. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 7, wherein characterized 

in that during fehe structuring ef- the concept dictionary 

includes : — 

determining navigation indicators -a^ee determined — from a 

matrix M = [c lf c 2 , c N ] e <R p * N -of the set C of concepts c ± e W 
where c ± represents a concept of p values, by implementing the 
following steps: 

i) calculating a representative w of the matrix M; 

ii) calculating afehe covariance matrix M between tke 
elements of the matrix M and the representative w of the matrix M; 

iii) calculating a projection axis u for projecting the 
elements of the matrix M; 

iv) calculating atfee value pi = d(u,Ci) - d(u,w) and 
decomposing the set C_of concepts [ [€] ] into two subsets JCl and 
C2]_ as follows: 

Jc, g CI if pi < 0 
|c ± g C2 if pi > 0 

v) storing a data set fefee — information {u, w, |pl|, p2 } in 
afehe node associated with the set C of concepts , where pi is atfee 
maximum of all pi < 0 and p2 is at-he- minimum of all pi > 0, the 
data set {u, w, |pl|, p2 } constituting the navigation indicators 
in the concept dictionary. 



10. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 1, characterized in that 
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both — the — structural components and the — complements of said 
structural components constituted by the — textural components of 
thean- image of the multimedia document are analyzed, and in that: 

a) while analyzing the structural components of the image: 

al) boundary zones of tte image structures are 

distributed into different classes depending on ant-he orientation 
of a the local variation in intensity so as to define structural 
support elements (SSEs) of the image; and 

a2) performing statistical analysis to construct terms 
constituted by vectors describing the — local properties and the 
global properties of the structural support elements; 

b) while analyzing the textural components of the image: 

bl) detecting and performing parametric characterization 
of a purely random component of the image; 

b2) detecting and performing parametric characterization 
of a periodic component of the image; and 

b3) detecting and performing parametric characterization 
of a directional component of the image; 

c) grouping athe set of descriptive elements of the image in 
a limited number of concepts constituted firstly by the — terms 
describing the local properties and the global properties of 
structural support element and secondly by the parameters of the 
parametric characterizations of the random, periodic, and 
directional components defining the textural components of the 
image; and 

d) for each document, defining a fingerprint from the 
occurrences, the — positions, and the — frequencies of said limited 
number of concepts . 
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11. (CURRENTLY AMENDED) An indexing mctho dA method of indexing 
multimedia documents according to claim 10, characterized in that 
the local properties of the structural support elements taken into 
consideration for constructing terms comprise at least one of t -he 
support types selected from amongst a linear strip or a curved arc 
as a support , the length and width dimensions of the support, at-he 
main direction of the support, and at-he shape and the statistical 
properties of the pixels constituting the support. 

12. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 10, characterized in that 
the global properties of the structural support element taken into 
account for constructing terms comprise at least afe-he number of 
each type of support and afeke spatial disposition thereof. 

13. (CURRENTLY AMENDED) An indexing mcthod A method of indexing 
multimedia documents according to claim 10, wherein charactcrizcd 
in that during analysis of the structural components of the image, 
a prior test is performed to detect whether at least one structure 
is present in the image, and in the absence of any structure, the 
method passes directly to the step of analyzing the textural 
components of the image. 

14. (CURRENTLY AMENDED) An indexing mcthod A method of indexing 
multimedia documents according to claim 10, characterized in that 
in order to distribute decompooc boundary zones of the image 
structures into different classes, starting from athe digitized 
image defined by athe set of pixels y(i,j) where (i,j) e I x J, 
where I and J designate^ respectively^ a thc number of rows and 
athe number of columns of the image, afehe vertical gradient image 
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g v (i,j) where (i,j) SIX J_^_ and a thc horizontal gradient image 
g h (i,j) with (i,j) e I x J are calculated, and the image is 
partitioned depending on athe local orientation of its gradient 
into a finite number of equidistant classes, the image containing 
the local orientation of itsfche gradient being defined by the 
equation: 



the classes constituting support regions likely to contain 
significant support elements are identified, and based on the 
basis — &§ — the support regions, significant support elements are 
determined and indexed using predetermined criteria. 

15. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 1, characterized in that 
while indexing a multimedia document comprising video signals, 
terms t ± are selected that are constituted by key-images 
representing groups of consecutive homogeneous images, and 
concepts c ± are determined by grouping together terms t ± . 

16. (CURRENTLY AMENDED) An indexing mcthod A method of indexing 
multimedia documents according to claim 15, characterized in that 
in order to determine key-images constituting terms t ± , a score 
vector SV is initially generated comprising a set of elements 
SV(i) representative of afehe difference or similarity between a 
first the content of an image of index i and a second the content 
of an image of index i-1, and the score vector SV is analyzed in 
order to determine key-images which correspond to maximums of the 
values of the set of elements SV(i) of the score vector SV. 
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17. (CURRENTLY AMENDED) An indexing mctho dA method of indexing 
multimedia documents according to claim 16, characterized in that 
an image of index j is considered as being a key-image ifj_ 

— afche value SV(j) of athe corresponding element of the score 
vector SV is a maximum and the value SV(j) is situated between two 
minimums minL and minR, and 

a if the minimum Ml given by ouch that 

Ml = (|SWS (j) - minL |, I SV (j) - minR|) 
is greater than a given threshold. 

18. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 1, characterized in that 
while indexing a multimedia document comprising audio components, 
the multimedia document is sampled and decomposed into frames, 
which frames are further subsequently grouped together into clips 
that are each being characterized by a— terms t ± constituted by a 
parameter vector. 

19. (CURRENTLY AMENDED) An indexing mcthod A method of indexing 
multimedia documents according to claim 18, characterized in that 
each [ [e] ] frame comprises about 512 samples to about 2,048 samples 
of afehe sampled audio document. 

20. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 18, characterized in that 
the parameters taken into account to define the terms t ± comprise 
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time information selected from the group consisting 
of corresponding to at least one of the following parameters : anfefee 
energy level of the audio signal frames, athe standard deviation 
of frame energ y levels^ ee- in the clips, at-he sound variation 
ratio, at-he low energy ratio, atfee rate of oscillation about a 
predetermined value, athe high rate of oscillation about a 
predetermined value, afehe difference between at-he number of 
oscillation rates above and below the mean oscillation rate for 
the frames of the clips, athe variance of the oscillation rate, or 
at-he ratio of silent frames. 

21. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 18, characterized in that 
the parameters taken into account for defining the terms t ± 
comprise frequency information selected from the group consisting 
of : corresponding to at leapt one of the following paramctcro! 

ajfc-ke center of gravity of at-he frequency spectrum of athe 
short Fourier transform of an/the audio signal, afeke bandwidth of 
the audio signal, athe ratio between an t -he — energy level in a 
frequency band to athe total energy level in the entire frequency 
band of the sampled audio signal, at-he mean value of spectrum 
variation of two adjacent frames in a clip, and a fe&e cutoff 
frequency of a clip. 

22. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 18, characterized in that 
the parameters taken into account for defining the terms t ± 
comprise at least energy modulation at 4 Hz. 
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23. (CURRENTLY AMENDED) An indexing mctho d A method of indexing 
multimedia documents according to claim 1, characterized in that 
the shape [ [■&] ] of the-aft image of the- a multimedia document are 
analyzed using the following steps: 

a) performing mult iresolution followed by decimation of the 
image; 

b) defining the image in polar logarithmic space; 

c) representing afehe query image or an image portion by its 
Fourier transform H; 

d) characterizing the Fourier transform H as follows: 

dl) projecting H in a plurality of directions to obtain 
a set of vectors of dimension equal to afche projection movement 
dimension; and 

6.2) calculating the — statistical properties of each 
projection vector; and 

e) representing the shape of the image by a term [ [fc^] ] 
constituted by values for the statistical properties of each 
projection vector. 
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